#making a ray tracer is just the side effect of me learning how to use compute shaders

[bobacupcake]

ahh .....

[thelordofdarkreunion]

Things the Scoundrels are No Longer Allowed to Do- Part II

Based on “Skippy’s List,” I have continued “A List of Things the Scoundrels are No Longer Allowed to Do.”  I hope you enjoy it.  The original can be found here:

https://thelordofdarkreunion.tumblr.com/post/637424500291600384/a-list-of-things-the-scoundrels-are-no-longer

The group known as the Magnificent Scoundrels has gotten a bit out of hand.  This list was compiled by Admiral Hackett of the Systems Alliance, Admiral Kelly of the GA, Fleet Admiral Hood of the UNSC, Inquisitor Vail of the Holy Inquisition, Commander Briggs of the Frontier Militia, Princess Leia of the New Republic, and Director Fury of SHIELD in order to curb the Scoundrels’ more dangerous or inappropriate behaviors.  These rules apply to all Scoundrels and their teams/crews.

207.  Expended ammunition is not a business expense.

208.  The Scoundrels are no longer allowed to call requesions officers or quartermasters “sugar daddy.”

209.  There is no “anti-Shepard conspiracy” within the Scoundrels’ fleet.  That’s the Citadel Council’s thing.

210.  The Scoundrels are no longer allowed to adopt dogs to “sic on the brass.”

211.  The Scoundrels are no longer allowed to pay Revenant to kill people they don’t like.

212.  None of the Scoundrels are The Chosen One.  That was Anakin Skywalker.

213.  It is wrong to fire warning shots at drivers who do not recognize your right of way.

214.  Reading is not “for officers only.”

215.  The Scoundrels are no longer allowed to ask anyone who outranks them if they’ve been smoking crack.

216.  The Scoundrels are no longer allowed to turn their starship command rooms into throne rooms.  Especially with tacky carpets.

217.  We are not making clones out of any of you.  You are all hard enough to deal with as is.

218.  The Scoundrels are no longer allowed to steal any massive, mobile space stations or star fortresses, which include but are not limited to:

- The Rock

-The Phalanx

-The Citadel

-High Charity

-Cloud City

219.  Thomas Drake is not allowed to crash economies “because it’s fun.”

220.  The Scoundrels are no longer allowed to assign nicknames to anyone.

221.  The Scoundrels are no longer allowed to make chain guns that fire miniature nukes

222.   The weapons specialists of the Apocalypse are no longer allowed to collaborate with the engineers of the Normandy or Enterprise, and Quill is to give up the nuke chain gun.

[I will not!  How can you stop me?  I have a chain gun that fires nukes!]

223.  The Scoundrels are no longer allowed to wake up superior officers with cymbals

224.  Napalm Sticks to Kids is not a motivational song.

225.  The Scoundrels are no longer allowed to quote bastardized versions of Dr. Seuss rhymes on military operations.

226.  Command decisions do not need to be ratified by a ⅔ majority.

227.   The Scoundrels are no longer allowed to line their helmets with tin foil to “block out the space mind control lasers.”

228.  The Scoundrels are no longer allowed to start any SITREP (Situation Report) with “I recently had an experience I just had to write you about…”

229.  Do not attempt to take the gas masks off of Death Korps troopers.

230.  Rodents are not entitled to burial with full military honors, even if they are “casualties of war.”

231.  The Scoundrels are no longer allowed to mock command decisions in front of the press.

232.  You should not speculate on the penis size of anyone who outranks you.  Especially if they’re in earshot.

233.  You cannot arrest children for being rude.

234.  The Scoundrels are no longer allowed to create new, made-up government forms, then insist they be filled out.

235.  No one is allowed to perform “lap dances” in uniform.

236.  Past lives have absolutely no effect on the chain of command.

237.  Cain is technically allowed to kill any of you if he finds reason to, so stop pissing him off.

238.  The Scoundrels are no longer allowed to defect to other military service branches during training missions.

239.  Your race is not “other” on official documents.

240.  There is no Scoundrels ethics committee.  And if there was, Thomas Drake would not be chairman.

241.  Chainsaws are not the answer to every question.  Nor is “more chainsaws.”  Or “chainsaw cannons.”  Except for that one time, and yes, it was awesome.

242.  Stop posting classified information on social media.

243.  Adam Vir is no longer allowed to play “Hippocratic Oath chicken” with Dr. Kril.

244.  The Scoundrels are no longer allowed to cook nerve gas in the sink.

245.  There is no “annoy” setting on a phaser.

246.  A wet towel is not an improvised weapon.  Unless you’re Master Chief.  There’s a reason the Covenant calls him “The Demon.”

247.  I know you all have passes, but if the gun can’t fit through the x-ray machine, it doesn’t go on the plane.

248.  The Scoundrels are no longer allowed to download Internet Explorer into the Geth hivemind or the Martian noosphere.

249.  The Scoundrels are no longer allowed to demand payment in liquor, backrubs, or bubble wrap.

250.  Any Exterminatus-grade weapon is not “my little friend.”

251.  Airlocks do not double as waste disposals.

252.  No member of the Scoundrels or their crews are a pagan god or goddess of fertility.

253.  The Scoundrels are no longer allowed to learn profanities in any language that can bend reality.

254.  The Scoundrels are no longer allowed to steal their own souls.

255.  There is not a Space Marine Chapter whose heraldry is a smiley face.

256.  The following weapons are no longer allowed as dueling choices: steamrollers, nerve gas, land mines, or heavy artillery.

257.  Shepard is no longer allowed to drive or pilot anything.

258.  Han Solo is no longer allowed to attempt any piloting maneuver in which the original inventor was killed doing.

259.  The Scoundrels are no longer allowed to play The Only Thing They Fear is You every time a super soldier enters the battlefield.

260.  In formal introductions to nobility, you are not allowed to introduce your companions as “the other guys.”

261.  The Scoundrels are no longer allowed to monologue.

262.  The Scoundrels are no longer allowed to do anything that would make a Sith Lord cry.

263.  The Scoundrels are no longer allowed to get telepaths to hurry up the speeches of long winded politicians.

264.  The Scoundrels are no longer allowed to hand over annoying reporters to any organization that could be considered a theocracy.

265.  If a black op requires you to impersonate an employee, you are not allowed to bill the target for overtime.

266.  By definition, chaplains cannot be atheist.

267.  The proper response to the question “Why?” is not “Why not?”

268.  It is assumed that a properly trained Titan Pilot knows what at least one of the buttons in the Titan’s cockpit does, and it is wrong for Cooper to pretend otherwise.

269.  At the end of a high profile assassination mission, the Scoundrels are no longer allowed to play disco music on the target’s phone.

270.  The Scoundrels cannot hear the soundtrack.

271.  Thermonuclear hand grenades do not exist, and the Scoundrels are no longer allowed to try and make them.

272.  I assure all of the Scoundrels with absolute certainty that Ralph is not a traditional Japanese name.

273.  None of the Scoundrels are from Margaritaville.

274.  Hawaiian shirts are not part of any of our governments’ formal uniforms.

275.  Master Chief is not allowed to record Gravemind ASMR.

276.  The Scoundrels are not allowed to write tell-all books about anything.

277.  “Legends never die!” is not a valid excuse.

278.  The Scoundrels are no longer allowed to think of new, creative, or fun uses for cursed artifacts.

279.  Check the door means listen to see if there’s any activity on the other side, not put multiple rounds through it.

280.  The Scoundrels are no longer allowed to give wasabi to unsuspecting aliens.

281.  The Scoundrels are no longer allowed to attempt to do anything they saw Jackie Chan do.

282.  The Scoundrels are no longer allowed to implement any battle plan that includes the words “and hope they miss a lot.”

283.  There is an upper limit to the number of people a bullet can go through.

284.  The Scoundrels are no longer allowed to attempt to unionize the Unggoy.

285.  The Scoundrels are no longer allowed to hack forge world PA systems so they only play Allentown.

286.  Sarcasm is wasted on Imperial Stormtroopers.

287.  The Scoundrels are no longer allowed to upload porn to the HUDs of their commanding officers.

288.  The Scoundrels are no longer allowed to download porn from the HUDs of their commanding officers.

289.  No matter how tough the battle, the Scoundrels are to keep the congratulatory ass-slapping to a minimum.

290.  The Scoundrels are no longer allowed to use time machines to invade Germany on August 31, 1939 and thus secure Belgian dominion over Europe.

291.  The Scoundrels are no longer allowed to Tokyo drift tanks

292.  The Scoundrels are no longer allowed to “catch air” in military vehicles.

293.  The Scoundrels are no longer allowed to send anything to the past, future, or alternate dimensions.

294.  The Scoundrels are no longer allowed to mount speakers on tanks to play Ghost Division as they drive into battle.

295.  The Scoundrels are no longer allowed to post memetic hazards on the internet.

296.  The Scoundrels are no longer allowed to bind eldritch dieties to their will and make them mow the lawn.

297.  The Scoundrels are no longer allowed to load weapons with all tracer rounds

298.  If your personal weapon can be read with a Geiger counter, you aren’t allowed to have it.

299.  The Scoundrels are no longer allowed to turn Khornite daemon worlds into self supporting blood banks.

300.  “Pimp my Death Star” is not a real show, and we are not bringing Grand Moff Tarkin back from the dead to host it.

301.  Prussian Glory March is not a disco song.

302.  We know that Shepard was brought back from the dead by Cerberus, but no matter how high profile or how close a friend, the Scoundrels are not allowed to ask Cerberus, the Adeptus Mechanicus, or, god forbid, Fabius Bile to bring anyone or anything back from the dead.

303.  Any weapon that can be set to “flay” is strictly forbidden.

304.  The Scoundrels are no longer allowed to sing the Oompa Loompa song every time someone annoying dies.

305.  The Scoundrels are no longer allowed to store squeeze tubes of explosive putty in medicine cabinets.

306.  On most planets, shoulder holsters are frowned upon as casual attire.

307.  Zero body count does not mean just the ones they can find.

308.  Walmart is not a one stop shopping place for hunting demons.

309.  The Scoundrels are no longer allowed to play football/soccer with AT-ST or Sentinel walkers.

310.  None of you are currently parents, but if you ever become one, Trazyn the Infinite is not to be named your child’s godfather.

311.  You know what, the Scoundrels are no longer allowed to name any of the following as godparents of any potential children:

-The God-Emperor of Mankind

- Emperor Palpatine

- Councilor Sparatus

- Leman Russ [Bjorn said it was OK so fuck you.]

- Kahless the Unforgettable

- Kuben Blisk

- Kharn the Betrayer

312.  Searching a building means entering it, not leveling it with artillery and digging through the rubble.

313.  FedEx does not deliver to Tatooine.

314.  None of the Scoundrels are allowed to single-handedly make Starfleet Academy the number one party school in the universe.

315.  Covering fire does not include nuclear weapons.

316.  The Scoundrels are no longer allowed to challenge anyone to a dance-off to the death.

317.  Kirk, rifts in the time-space continuum are not for your personal amusement.

318.  Blowing up the top twenty floors of a building is not a “diversion.”

319.  The Scoundrels are no longer allowed to convince Sith Lords to use Force lighting on their welding projects.

320.  Canadian is not a real language, and you can’t set your translators to it.

321.  There is no such thing as a were-saxophonist.

322.  The Scoundrels are no longer allowed to tell new personnel that starship windows can be rolled down.

323.  The Scoundrels are no longer allowed to start wars between major weapons corporations, especially “because I’m bored.”

324.  The Scoundrels are no longer allowed to clear enemy underground bunker complexes just using Bangalore torpedoes.

325.  The Scoundrels are no longer allowed to order a lance of Imperial Knights to perform synchronized dance numbers.

326.  The Scoundrels are to leave out human mating rituals when presenting cultural exchanges to alien ambassadors.

327.  When raiding enemy corporations or terrorist organizations, the Scoundrels are no longer allowed to look at the target’s HR files to see if they have better benefits.

328.  The Scoundrels are no longer allowed to use time machines to collect autographs.

329.  Any buttocks belonging to the Scoundrels or any of their crews are permanently forbidden from making contact with any copy machine.

330.  The Scoundrels are no longer allowed to go FTL to avoid red lights.

331.  “Just throw them out the airlock” is not a backup first contact protocol.

332.  The Scoundrels are no longer allowed to use the Enterprise’s transporters to fill enemy starships with jello.

333.  None of the Scoundrels are the patron saints of large explosions.

334.  The Scoundrels are no longer allowed to make, accept, or take rake-offs on bets concerning X-class end-of-the-universe scenarios.

335.  The Scoundrels are no longer allowed to go on PA systems and announce they just won The Game.  Goddammit.

336. The Scoundrels are no longer allowed to trick Captain Marvel and Cato Sicarius any superheroes or super soldiers they deem “annoying” into fighting each other.

337.  The Scoundrels are no longer allowed to sell tickets to or organize cage matches between prominent super soldiers.

338.  Lockpicking and door breaching are two entirely different things.

339.  Performing obscene acts while in the cockpit of or piloting large combat mechs is strictly prohibited.

340.  Freeing slaves out of justice is good.  Out of spite, not so much.

341.  The Scoundrels are no longer allowed to compose offensive emails during stealth operations on the target CEO’s email and subsequently CC the entire company.

342.  The Scoundrels are no longer allowed to recreate the Charge of the Light Brigade with the Death Riders of Krieg.

343.  The Scoundrels are no longer allowed to use telepaths to get out of speeding tickets.

344.  The state-controlled news service of the Imperium of Man most definitely does not have a liberal bias.

345.  Likewise, the state-controlled news service of the United Federation of Planets does not have a conservative bias.

346.  The Scoundrels are no longer allowed to kidnap Ewoks or Volus and put them in hamster wheels.

347.  Adam Vir is to, by order of Supreme Grand Master Azrael of the Dark Angels, return the Watchers in the Dark he took from The Rock as pets.

348.  The Scoundrels are no longer allowed to spend the entirety of their bonus pay on lottery tickets.

349.  The very concept of a Hutt lap dancer will earn a surprise visit from the Deathwatch.

350.  The Scoundrels are no longer allowed to use grenade launchers to play bocce ball.

351.  If you are unsure of which side of the road you are supposed to drive on, the middle of the road is not a healthy compromise.

352.  No matter how cool it would be, the Scoundrels are not allowed to use any time machine to loan General Eisenhower a squadron of X-wings for D-Day.

353.  The Scoundrels are no longer allowed to program medical droids for “aggressive dentistry.” 

354.  The Scoundrels are to stop trying to get a reality TV show based on themselves.

355.  Garrus Valkarian is not “on loan” to the Vindicare Temple to improve either his or their sniping skills.

356.  Pointing out a massive plothole in any bad guy’s plan will not stop them from attacking you.

357.  Preliminary nuclear bombardment is not automatically Plan A.

358.  Maverick and Tope are not tax exempt for being chaplains. 

359.  Thomas Drake is to stop teaching classes to the rest of the Scoundrels on tax evasion.

360.  The Scoundrels are no longer allowed to refit tanks with jump jets.

361.  The Scoundrels are no longer allowed to create their own currencies.

362.  The Scoundrels are no longer allowed to attempt to set Jawas on fire with a massive magnifying glass.

363.  The Stanley Cup does not have the same power as the Holy Grail.  Not even on Canadians.

364.  The Scoundrels are not allowed to steal the Stanley Cup.

365.   The Eldar really hate it when you greet them with “Live long and prosper.”

366.   The Scoundrels are no longer allowed to give pre-teen children their phone numbers, especially when they are on black ops.

367.  You cannot partake in the sport of fencing with a broadsword.

368.  The Scoundrels are no longer allowed to switch nationalities or service branches for tax purposes.

369. None of the Scoundrels are “He who must not be named only in passing.”

370.  The Scoundrels are no longer allowed to shoot at natural disasters.

Well, there it is.  I hope you enjoyed it, and if you would like to add to the list, feel free!  

[descubes]

In the Paris Open Source Summit, I had a long discussion with engineers from AdaCore, that reminded me of the early roots of XL in Ada. I have never really retraced the steps along the way, and this was an interesting walk for me. I thought I’d share…

My interest in programming languages is quite old. Actually, I had been writing development tools for as long as I can remember:

My first published program, at age 14, was an extension of the Sinclair Spectrum BASIC that gave new features such as flood fill or a primitive window system.

A few years later, I published HPDS (HP Development System), a cross-compiler for HP-48 and HP-28 calculators that used an extended version of the built-in language, including a complete assembler (you can already see the Alsys commenting style in this example). HPDS did not have much success, but some of the games I developed with it are still available on the Internet today.

Also, I had been deeply fascinated by the Sinclair QL’s SuperBasic, which had departed so much from regular BASIC that it was probably closer to Pascal.

The direction of my thinking about programming languages changed drastically when, as an engineering student, I met the Ada programming language.

Step 0 (1990s): From Alsys SA to LX

During my last year in engineering school, my internship was at Alsys SA, a company that was producing high-quality Ada compilers, written themselves in Ada. I have very fond memories of that period. I learned quite a bit with the brilliant engineers at that company. The commenting style I still use today is a remnant of that training period.

I learned Ada by reading the reference manual. I remember being very impressed by this language, notably by the standardisation effort. At the time, no two Pascal or C compilers were behaving the same. You had the Turbo Pascal dialect, the Think C dialect which was different from the MPW C, etc… I also liked the “solid feel” of the Ada language.

Yet, from the very beginning, I felt like Ada was a bit too restrictive. That was the incentive I needed to start thinking about my own programming language.

Step 0bis: WASHB – What Ada Should Have Been

In the very early days, I thought of my language as some enhanced version of Ada. So I called it WASHB, short for What Ada Should Have Been. Obviously, my knack for catchy names and acronyms had not entirely developed back then. That name obviously did not stick for very long, and WASHB was never anything more than a vague specification.

Step 1 (1995): LX (Langage eXpérimental)

The first serious effort at creating a real language was called LX, which were the initials for “Langage eXpérimental”, or “experimental language” in Shakespearian. I don’t have much data on the evolution of the language at that time, besides old Word documents that are unreadable on modern versions of Word. In particular, I no longer have source code that I can reliably trace back to that era.

  An LX compiler generating 68K assembly code

I do remember however that I had a compiler that was generating assembly code for the Motorola 68K family of CPUs. I was developing on Atari ST at the time, and I remember testing the generated code on an embedded board with a 68040 back when I was working for the HP Test and Measurement Organisation in France.

That early compiler went far enough to compile “Hello World“. I do believe that this was with a standard library written in the language itself.

A language that can build its own standard library

LX was still very Ada-like. Still, as I recall (I no longer have computer records of that period), several ideas had already solidified by that time, which I will expand on later:

Giving up on superfluous syntactic markers such as terminating semi-colon.

Using generics to write standard library component such as arrays or I/O facilities.

Making the compiler an integral part of the language, which led to…

having a normalised abstract syntax tree, and…

considering “pragmas” as a way to invoke compiler extensions.

I am quite positive about these ideas emerging at the time, because they derived from concerns about Ada having too many “magic thingies”, and these concerns were already well formed while I was working at Alsys.

What I disliked about Ada

I never liked magic in a language. To me, keywords demonstrated a weakness in the language, since they indicated something that you could not build in the library using the language itself. Ada had plenty of keywords and magic constructs.

Let me elaborate a bit on some specific frustrations with Ada:

Tasks in Ada were built-in language constructs. This was inflexible. Developers were already hitting limits of the Ada-83 tasking model. My desire was to put any tasking facility in a library, while retaining an Ada-style syntax and semantics.

Similarly, arrays were defined by the language. I wanted to build them (or, at least, describe their interface) using standard language features such as generics. Use cases I had in mind was interfacing with languages that had different array layouts such as Fortran and C, or using an array-style interface to access on-disk records. Back then, mmap was unavailable on most platforms.

Ada text I/O facilities were uncomfortable. But at that time, there was no good choice, and it was mostly a game of picking the poison that would kill you:

In Pascal, WriteLn could take as many arguments as you needed and was type safe, but it was a magic procedure in Pascal, that you could not write yourself in standard Pascal, nor extend or modify to suit your needs.

Ada’s text I/O facilities only took one argument at a time, which made writing the simplest I/O statement quite tedious relative to C or Pascal.

C’s printf statement had multiple arguments, but was neither type safe nor extensible, and the formatting string was horrid.

I also did not like pragmas, which I found too ad-hoc, with a verbose syntax. I saw pragmas as indicative that some kind of generic “language extension” facility was needed, although it took me a while to turn that idea into a reality.

From experimental to extensible

I soon realised that my efforts were mostly about being able to extend the language through its standard library and mechanisms such as pragmas. At some unspecified point in time, somewhere along the way, the meaning of LX changed from experimental language to extensible language. I liked the normalisation of Ada, but I wanted a way to leverage the base language to go beyond the basics in a controlled and specified way.

The development period for LX lasted between my training period at Alsys in 1990 and 1998, when I jointed the HP California Language Lab in Cupertino (CLL) to work on the C++ compiler and, I hoped, my own language. It did not all go as planned…

Step 2 (1998): XL, meet Xroma

One of the very first things I did moving to the US was to translate the language name to English. So LX turned into XL. This was a massive rename in my source code, but everything else remained the same.

Daveed Vandevoorde and meta-programming

As soon as I joined the CLL, I started talking about my language and the ideas within. One CLL engineer who immediately “got it” is Daveed Vandevoorde. Daveed immediately understood what I was doing, in large part because he was thinkering along the same lines. He pointed out that my approach had a name: meta-programming, i.e. programs that deal with programs. I was doing meta-programming without knowing about the word, and I felt really stupid at the time, feeling that everybody knew about that technique but me.

Daveed was very excited about my work, because he was himself working on his own pet language named Xroma (pronounced like Chroma). At the time, Xroma was, I believe, not as far along as XL, since Daveed had not really worked on a compiler. However, it had annotations similar to my pragmas (which I suspect are distant ancestors of C++11 attributes), and some kind of public representation for the abstract syntax tree as well.

Also, the Xroma name was quite Xool, along with all the puns we could build using a capital-X pronounced as “K” (Xolor, Xameleon, Xode, …) or not (Xform, Xelerate, …). As a side note, I later called “Xmogrification” the VM context switch in HPVM, probably in part as a residual effect of the Xroma naming conventions.

In any case, Daveed and I joined forces, that combined effort was named Xroma. I came up with the early version of the lightbulb logo, and Daveed did a nice 3D rendering of the same using the Persistence of Vision ray tracer.

Concept  programming

The discussions around our respective languages, including the meta-programming egg-face moment, led me to solidify the theoretical underpinning of what I was doing with XL. It actually did go somewhat beyond meta-programming, which was really only a technique being used, but not the end goal. I called my approach Concept Programming. I tried to explain what it is about in this presentation.

Concept programming is about how we transform concepts that reside in our brain into code that resides in the computer. That conversion is lossy, and concept programming explores various techniques to limit the losses. It introduces pseudo-metrics inspired by signal processing such as syntactic noise, semantic noise, bandwidth and signal/noise ratio. More importantly, Concept Programming has consistently guided what I am doing with XL.

From examples of concept programming to C++ concepts

As an aside, talking about concepts, I have reasons to suspect that C++ concepts might be the result of a mis-interpretation of several concept-programming e-mail discussions I had with a few C++ committee members around the year 2000. Maybe I’m wrong and it’s just a coincidence. Besides the emails I sent, there is some striking similarity.

Consider the following concept example from Wikipedia:

template <class T> concept bool EqualityComparable() { return requires(T a, T b) { {a == b} -> Boolean; // Boolean is the concept defining a type usable in boolean context {a != b} -> Boolean; }; }

To me, it looks very much like a C++ version of this example (called validated generics in XL), which I was using as an illustration of the outcomes of concept programming in practically any discussion of these topics back in 2000:

generic type ordered where A, B : ordered Test : boolean := A < B

As of this writing (December 2017), C++ concepts did not yet make it to the C++ standard. By contrast, a variant of the code above shows up on a web page that states “First published on February 17, 2000“. Whether reinvented independently or not, this specific idea, that I will keep calling validated generics, took about 20 years to make it to C++.

The one thing that makes me unhappy about C++ concepts is the name concepts. In Concept Programming, concepts reside in your head, never in the code. So calling something in the code “concept” is bound to make concept programming much harder to explain to developers who learned C++ concepts first.

A program database to rule them all

Daveed is still a prominent and innovative member of the C++ community today. Back in the CLL days, he was already quite influential, being for example the HP representative to the C++ Standard committee.

He quickly generated quite a bit of interest in the CLL about some kind of universal program database that would represent the program in such a way that various tools could work on it. We called such transformations thin tools. The database format was intended to work across languages, so it was designed to be able to represent a C++ parse tree or a Java parse tree in a very similar way. There was tremendous interest about this kind of technology at the time, and like many things, these ideas took years to materialise elsewhere.

Daveed and I gave several internal talks about our ideas, and I was happy to let him speak if only because my english accent at the time was much worse than his. That proved to be a mistake…

Switching to the off-side rule

Another major visual change that happened around that time was switching to the off-side rule, i.e. using indentation to mark the syntax. Python, which made this approach popular, was at the time a really young language (release 1.0 was in early 1994).

Alain Miniussi, who made a brief stint at the CLL, convinced me of giving up the Ada-style begin and end keywords. I was initially totally unconvinced, and reluctantly tried it on a fragment of the standard library. As soon as I tried it, however, the benefits immediately became apparent. It was totally consistent with the core idea of concept programming, namely that the code should look like your concepts. Enforcing indentation made sure that the code did look like what it meant.

It took some effort to convert existing code, but I’ve never looked back since then.

Daveed leaves HP and XL loses traction

Not very long after I joined the CLL, Daveed received one of these offers that you can’t refuse from the Edison Design Group, the company he’s still working for to this day. So he left HP. Before leaving, he asked me to keep the Xroma name for his own project, and together, we decided that I would rename my side as Mozart. I was disappointed, because I liked the name Xroma, but did not think too much of it.

However, it quickly became evident that after Daveed’s departure, XL had lost all traction in the CLL. The next talk I organised about it had maybe two people. Clearly, CLL engineers saw Xroma as being solely Daveed’s baby, with me just tagging along. It was obvious that everybody thought the project had died the day Daveed left HP.

Trying Apple, unsuccessfully

After a few months without any traction at HP, I started looking elsewhere. Through some chance event, I ended up pitching XL to Bertrand Serlet at Apple. I rarely met someone so smart. I remember being vastly impressed that the second or third question he asked was “How do you deal with introspection“. I did not know at the time how important introspection was in the Objective-C programming model. That question, coming from a higher-up at Apple, was totally unexpected. It convinced me that Apple knew what they were talking about.

In any case, Bertrand Serlet was convinced enough that I interviewed with the compiler team at Apple. Things went quite well, until someone asked “Are you sure you own this language?” I personally had absolutely no doubt about that, having invented that language long before even joining HP. However, the Apple engineer who had asked the question was quick to point out that if I had spoken about it to HP engineers, California Work for Hire agreements probably meant that HP had taken ownership of my work.

So the discussion with Apple stopped abruptly, and I returned to HP somewhat bummed. I still did a few things at the CLL, like porting the HP C++ compiler to Itanium or representing HP at the C++ committee for a couple of years. But I had no real love for C++, and I hit a career wall within the CLL. I finally left the C++ compiler team after only two years, and started HP Integrity Virtual Machines. But that is another story.

Step 3 (2000): Open-sourcing XL and Mozart

Before leaving the CLL, and following the Apple incident, I talked with various people at HP to make sure my intellectual property of XL was acknowledged.

It took quite a bit of time, but we reached an agreement as follows: XL and Mozart had to be made open-source, and published regularly so that HP could monitor whether they were interested in it. Also, because HP had a strong interest in Java at the time, they wanted to make sure that whatever I developed also worked for Java.

Moka, a Java to Java compiler

So I published Mozart and began working actively on it, making sure there was some prominent Java support in it. I also published an article in Dr Dobb’s, a popular developer journal.

But my heart was never with Java anymore than with C++, as evidenced by the much more extensive documentation about XL on the Mozart web site. As a language, Java had very little interest for me.

Key innovations in 2000-vintage XL

By that time, XL was already quite far away from the original Ada. Here are some of the key features that went quite a bit beyond Ada:

The syntax was quite clean, with very few unnecessary characters. There were no semi-colons at the end of statement, and parentheses were not necessary in function or procedure calls, for example. The off-side rule I talked about earlier allowed me to get rid of any begin or end keyword, without resorting to C-style curly braces to delimit blocks.

Pragmas extended the language by invoking arbitrary compiler plug-ins. As I already pointed out, I suspect that attributes in C++11 are distant (and less powerful) descendants of this kind of annotation, if only because their syntax matches my recollection of the annotation syntax in Xroma.

Expression reduction was a generalisation of operator overloading that works with expressions of any complexity. To this day, expression reduction still has no real equivalent in any other language that I know of, although expression templates can be used to achieve similar effect in a very convoluted way for expressions following the standard operator syntax in C++.

True generic types were a way to make generic programming much easier by declaring generic types that behaved like regular types. Validated generic types extended the idea by adding a validation to the type, and they also have no real equivalent in other languages that I am aware of, although C++ concepts bring a similar kind of validation to C++ templates.

Type-safe variable argument lists made it possible to write type-safe variadic functions. They solved the WriteLn problem I referred to earlier, i.e. they made it possible to write a function in a library that behaved exactly like the Pascal WriteLn. I see them as a distant ancestor of variadic templates in C++11, although like for concepts, it is hard to tell if variadic templates are a later reinvention of the idea, or if something of my e-mails influenced members of the C++ committee.

A powerful standard library was not quite there, but the key foundations were there, and it was mostly a matter of writing it. My implementation of complex numbers, for example, was 70% faster than C++ on simple examples, because it allowed everything to be in registers instead of memory.

There were a few things that I believe also date from that era, like getting rid of any trace of a main function, top-level statements being executed as in most scripting languages.

Limitations of the parse tree representation

One thing did not work well with Mozart, however, and it was the parse tree representation. That representation, called Notes, was quite complex. It was some kind of object-oriented representation with many classes. For example, there was a class for IfThenElse statements, a Declaration class, and so on.

This was quite complex, and made it extremely difficult to write thin tools, in particular thin tools that respected subtle semantic differences between languages. By 2003, I was really hitting a wall with XL development, and that was mostly because I was also trying to support the Java language which I did not like much.

One of the final nails in the Mozart coffin was a meeting with Alan Kay during an HP technical conference (he was an HP Fellow at the time). I tried to show him how my language was solving some of the issues he had talked about during his presentation. He did not even bother looking. He simply asked: “Does your language self-compile?��. When I answered that the compiler being written in C++, Alan Kay replied that he was not interested.

That gave me a desire to consider a true bootstrap of XL. That meant rewriting the compiler from scratch. But at that time, I had already decided that the internal parse tree representation needed to be changed. So that became my topic of interest.

Step 4 (2003): Bootstrapping XL2

The new implementation was called XL2, not as a version number, but because I was seeing things as a three-layer construction:

XL0 was just a very simple parse tree format with only eight node types.

XL1 was the core language, without any library.

XL2 was the full language, including its standard library.

This language is still available today, and while it’s not been maintained in quite a while, it seems to still pass most of its test suite. More importantly, the XL0 format has remained unchanged since then.

The XL0 parse tree format

The parse tree format is something that makes XL absolutely unique among high-level programming languages. It is designed so that code that can look and feel like an Ada derivative can be represented and manipulated in a very simple way, much like Lisp lists are used to represent programs.

The picture below shows an example of XL source code on the left, along with a graphical rendering of the internal representation on the right (click for details):

The parse tree format consists of only eight node types, four leaf node types (integer, real, text and symbol), four inner node types (infix, prefix, postfix and block). It is very vaguely documented here.

A few surprising properties of this parse tree format is that individual program lines are seen as the leaves of an infix “newline” operator. There are no keywords at all, the precedence of all operators being given dynamically by a syntax file.

Bootstrapping XL

The initial translator converts a simplified form of XL into C++ using a simplified form of transcoding. The simplified form of XL2 acceptable as input for this translation phase is only used in the bootstrap compiler. It already looks a bit like the final XL2, but error checking and syntax analysis are practically nonexistent.

The bootstrap compiler can then be used to translate the native XL compiler. The native compiler performs much more extended semantic checks, for example to deal with generics or to implement a true module system. It emits code using a byte-code that is converted to a variety of runtime languages. For example, the C bytecode file will generate a C program, turning the native compiler into a transcoder from XL to C.

That native compiler can translate itself, which leads to a true bootstrap where the actual compiler is written in XL, even if a C compiler is still used for the final machine code generation.

The XL2 compiler advanced to the point where it could pass a fairly large number of complex tests, including practically all the things that I wanted to address in Ada:

Pragmas implemented as compiler plug-ins.

Expression reduction generalising operator overloading.

An I/O library that was as usable as in Pascal, but written in the language and user-extensible.

A language powerful enough to define its own arrays or pointers, while keeping them exactly as usable as built-in types.

Compiler plugins

XL2 had full support for compiler plug-ins, in a way similar to what had been done with Mozart. However, plug-ins were much simpler to develop and maintain, since they had to deal with a very simple parse tree structure.

For example, the differentiation plugin implements symbolic differentiation for common functions. It is tested here. The generated code after applying the plugin would look like this. The plugin itself is quite simple. It simply applies basic mathematical rules on parse trees. For example, to perform symbolic differentiation on multiplications, the code looks like this:

function Differentiate (expr : PT.tree; dv : text) return PT.tree is translate expr when ('X' * 'Y') then dX : PT.tree := Differentiate(X, dv) dY : PT.tree := Differentiate(Y, dv) return parse_tree('dX' * 'Y' + 'X' * 'dY')

Meta-programming became almost entirely transparent here. The translate statement, itself provided by a compiler plug-in (see below), matches the input tree against a number of shapes. When the tree looks like X*Y, the code behind the matching then is evaluated. That code reconstructs a new parse tree using the parse_tree function.

Also notice the symmetric use of quotes in the when clause and in the parse_tree function, in both cases to represent variables as opposed to names in the parse tree. Writing parse_tree(X) generates a parse tree with the name X in it, whereas parse_tree('X') generates a parse tree from the X variable.

Translation extension

A particularly important compiler extension provided the translation and translate instructions. Both were used extensively to rewrite XL0 parse trees easily.

We saw above an example of translate, which translated a specific tree given as input. It simply acted as a way to compare a parse tree against a number of forms, evaluating the code corresponding to the first match.

The translation declaration was even more interesting, in that it was a non-local function declaration. All the translation X from all modules were accumulated in a single X function, several functions corresponding to distinct phases in the compiler. This made it possible to distribute translation XLDeclaration statements throughout the compiler, dealing with declaration of various entities, with matching translation XLSemantics for the later semantics analysis phase.

This approach made it quite easy to maintain the compiler over time. It also showed how concept programming addressed what is sometimes called aspect-oriented programming.

Step 5 (2009): Dynamic code generation

One issue I had with the original XL2 approach is that it was strictly a static compiler. The bytecode files made it possible to generate practically any language as output. I considered generating LLVM bitcode, but thought that it would be more interesting to use an XL0 input instead. One reason to do that was to be able to pass XL0 trees around in memory without having to re-parse them. Hence XLR, the XL runtime, was born.

XLR, the functional variant of XL

For various reasons, I wanted XLR to be dynamic, and I wanted it to be purely functional.  My motivations were:

a long-time interest in functional languages.

a desire to check that the XL0 representation could also comfortably represent a functional languages, as a proof of how general XL0 was.

an intuition that sophisticated type inference, Haskell-style, could make programs both shorter and more solid than the declarative type systems of Ada.

While exploring functional languages, I came across Pure, and that was the second big inspiration for XL. Pure prompted me to use LLVM as a final code generator, and to keep XLR extremely simple.

Translating using tree rewrites

As a matter of fact, I sometimes describe XLR as a language with a single operator, ->, which reads as transforms into. Thus, X->0 declares a variable X. This notation can be used to declare basic operators:

x:integer - y:integer as integer -> opcode Sub

It makes a declaration of writeln even shorter than it was before:

write x:text as boolean -> C elfe_write_text write x:integer as boolean -> C elfe_write_integer write x:real as boolean -> C elfe_write_real write x:character as boolean -> C elfe_write_character writeln as boolean -> C elfe_write_cr

More interestingly, even if-then-else can be described that way:

if true then TrueBody else FalseBody -> TrueBody if false then TrueBody else FalseBody -> FalseBody if true then TrueBody -> TrueBody if false then TrueBody -> false

Similarly for basic loops, provided your translation mechanism implements tail recursion properly:

while Condition loop Body -> if Condition then Body while Condition loop Body until Condition loop Body -> while not Condition loop Body loop Body -> Body; loop Body for Var in Low..High loop Body -> Var := Low while Var < High loop Body Var := Var + 1

Note that the fact that such structures can be implemented in the library does not mean that they have to. It is simply a proof that basic amenities can be constructed that way, and to provide a reference definition of the expected behaviour.

Step 6 (2010): Tao3D

When I decided to leave HP, I thought that XLR was flexible enough to be used as a dynamic document language. I quickly whipped together a prototype using XLR to drive an OpenGL 3D rendering engine. That proved quite interesting.

Over time, that prototype morphed into Tao3D. As far as the XLR language itself is concerned, there wasn’t as much evolution as previously. A few significant changes related to usability popped up after actively using the language. For example, implicit conversions of integer to real were not in the original XLR, but it was quite annoying in practice when providing object coordinates.

Tao3D developed a relatively large set of specialised modules, dealing with things such as stereoscopy or lens flares. As a product, however, it was never very successful, and Taodyne shut down in 2015.

Step 7 (2015): ELFE

ELFE is another application of XL’s extensibility to another application domain, namely distributed software. The idea was to take advantage of the existence of the XL0 standard parse tree to communicate programs and data across machines.

An ELFE program looks as as if it was running on a single machine, but actively exchanges program segments and their associated data between distant nodes. ELFE only adds a very small number of features to the standard XL:

The ask statement sends a program, and returns the result of evaluating that program as if it has been evaluated locally. It works like a remote function call.

An invoke statement sends a program to a remote node. It’s a “fire and forget” operation, but leaves a reply channel open while it’s executing.

Finally, the reply statement allows a remote node to respond to whoever invoke‘d it, by evaluating one of the available functions in the caller’s context.

A few very simple ELFE demos illustrate these remote-control capabilities. For example, it’s easy to monitor temperature on two remote sensor nodes, and to ask them to report if their temperatures differ by more than some specified amount.

ELFE was designed to run with a small memory footprint, so it provides a complete interpreter that does not require any LLVM. On the other hand, the LLVM support in that “branch” of the XL family tree fell into a bad state of disrepair.

And today?

These days, I find myself with several subtly distinct variants of XL which all share the same XL0, but have very different run-time constraints.

Languages using the XL0 parse tree format

Languages currently using the same XL0 parse tree format include;

Tao3D has the most advanced library, and a lot of code written for it. But that code often depends on undesirable behaviours in the language, such as implicit by reference argument passing.

ELFE has the most advanced type system of all variants, being able to perform overloading based on the shape of parse trees, and having a rather complete set of control structures implemented in the library. It also has an interesting modular structure, and a rather full-featured interpreter.

XLR fell behind with respect to LLVM support, LLVM not being particularly careful about release-to-release source compatibility. On the other hand, it had the most advanced type inference system, which allowed it to get performance that was close to C for simple cases.

XL2 has been left aside for a few years, but is still all but obsolete. It would need a bit of love to make progress with the standard library and actually connect the XLR back-end as initially envisioned.

Re-converging?

Overall, the effort today is to re-converge these various branches and to catch-up with LLVM. The ideal converged solution would have:

The modular structure, interpreter, remote control capabilities and type system developed for ELFE.

The real-time graphic capabilities of Tao3D, probably offered as an ELFE-style module

A finished Haskell-style type inference system

An XL2 front-end for those who prefer an imperative programming style

Future ideas

In addition, I have been toying with a few ideas for a while:

Using is instead of -> as the as the one-and-only rewrite operator. I believe that for most programmers, X is 0 is immediately understandable, whereas the current X->0 requires an explanation.

Replacing the block node type with an sequence or array node type.

Currently, blocks without a content, such as ( ) or { }, have a blank name inside, which I find ugly. It would make more sense to consider them as arrays with zero length.

Furthermore, blocks are often used to hold sequences, for example sequences of instructions. It would be easier to deal with a block containing a sequence of instructions than with the current block containing an instruction or a chain of infix nodes.

Bootstrapping an XLR compiler or interpreter, to validate that the XLR-level language is good enough for a compiler.

Conclusion

This article was way too long.

    From Ada to XL in 25+ years… In the Paris Open Source Summit, I had a long discussion with engineers from AdaCore, that reminded me of the early roots of…